Repository navigation
fix(replay-vision): trim repeated fields from scanner observation rows - #95794
posthog[bot] wants to merge 7 commits into
Conversation
The scanner-scoped observations list re-embedded the scanner's prompt and recordings query on every row, and restated each row's reasoning a second time as reasoning_segments. On the default page this overflowed an agent's token budget, so the tool could not be called without first guessing a small limit. Declare the three paths under `response.exclude` on the MCP tool, and default its `limit` to 20. The trim sits in the MCP layer because the app's own observations table, calibration tab, replay dock, and clipboard export all read those fields from list rows; the REST API keeps serving them. Point the tool description and agent note at vision-scanners-observations-get for one observation's full prompt and citation timestamps. Generated-By: PostHog Desktop Task-Id: cfd06794-5dc1-4cdb-b3d1-f66d05273694
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
🦔 PostHog Review reviewed this pull requestFound 0 must fix, 4 should fix, 1 consider. Published 5 findings (view the review). Resolved comments: 4 fixed, 1 declined |
🤖 CI report
|
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.50 MiB · 22 files | no change | ███░░░░░░░ 33.2% of 4.51 MiB |
logged-out boot: index + App + bootApp (preloaded by every page, including /login)src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts |
5.25 MiB · 668 files | no change | █████████░ 93.3% of 5.63 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
7.67 MiB · 2,677 files | no change | █████████░ 87.4% of 8.77 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 1000 B | src/index.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 854 B | src/scenes/ChunkLoadErrorBoundary.tsx |
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
| Size | File |
|---|---|
| 276.1 KiB | ../node_modules/.pnpm/posthog-js@1.427.2_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 255.2 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 104.4 KiB | src/lib/api.ts |
| 79.1 KiB | src/products.tsx |
| 68.8 KiB | src/lib/lemon-ui/icons/icons.tsx |
| 62.5 KiB | src/lib/utils/eventUsageLogic.ts |
| 38.8 KiB | ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js |
| 33.9 KiB | ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 276.1 KiB | ../node_modules/.pnpm/posthog-js@1.427.2_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 255.2 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 153.8 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 104.4 KiB | src/lib/api.ts |
| 95.8 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
| 79.1 KiB | src/products.tsx |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.28 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.28 MiB · 18 files | no change | ████░░░░░░ 39.9% of 5.72 MiB |
| Deferred (lazy) | 2.11 MiB · 45 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.1 KiB | no change | █░░░░░░░░░ 5.8% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 756.3 KiB | dist/toolbar/toolbar-app-4464UOOM.css |
| 603.1 KiB | dist/toolbar/chunk-chunk-NCE4QL3M.js |
| 484.7 KiB | dist/toolbar/chunk-chunk-D33R7XDZ.js |
| 136.5 KiB | dist/toolbar/chunk-chunk-BXI27EUT.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-FDH2IBXT.js |
| 72.2 KiB | dist/toolbar/toolbar-app-CJMMARKZ.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-TSAL54PB.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-VLYPUESV.js |
| 21.0 KiB | dist/toolbar/chunk-chunk-LPBZV4EM.js |
| 6.8 KiB | dist/toolbar/chunk-chunk-DV7IWQNF.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — no change
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1457.43 MiB · no change
ℹ️ MCP UI apps size — 33 app(s), 17665.2 KB JS
Built size of each MCP UI app (main.js + styles.css).
| App | JS | CSS |
|---|---|---|
| debug | 600.0 KB | 195.2 KB |
| action | 458.2 KB | 195.2 KB |
| action-list | 564.9 KB | 195.2 KB |
| cohort | 457.2 KB | 195.2 KB |
| cohort-list | 563.8 KB | 195.2 KB |
| email-template | 457.0 KB | 195.2 KB |
| error-details | 472.9 KB | 195.2 KB |
| error-issue | 457.9 KB | 195.2 KB |
| error-issue-list | 564.7 KB | 195.2 KB |
| experiment | 562.0 KB | 195.2 KB |
| experiment-list | 565.6 KB | 195.2 KB |
| experiment-results | 567.1 KB | 195.2 KB |
| feature-flag | 567.6 KB | 195.2 KB |
| feature-flag-list | 571.4 KB | 195.2 KB |
| feature-flag-testing | 461.4 KB | 195.2 KB |
| inline-scan | 457.7 KB | 195.2 KB |
| insight-actors | 563.0 KB | 195.2 KB |
| invite-email-preview | 456.4 KB | 195.2 KB |
| llm-costs | 560.0 KB | 195.2 KB |
| session-recording | 459.0 KB | 195.2 KB |
| survey | 458.8 KB | 195.2 KB |
| survey-global-stats | 562.7 KB | 195.2 KB |
| survey-list | 565.5 KB | 195.2 KB |
| survey-stats | 562.7 KB | 195.2 KB |
| trace-span | 457.6 KB | 195.2 KB |
| trace-span-list | 564.7 KB | 195.2 KB |
| vision-observation-list | 563.9 KB | 195.2 KB |
| workflow | 457.5 KB | 195.2 KB |
| workflow-list | 564.2 KB | 195.2 KB |
| loops-review | 461.9 KB | 195.2 KB |
| query-results | 756.3 KB | 195.2 KB |
| render-ui | 839.5 KB | 195.2 KB |
| visual-review-snapshots | 462.0 KB | 195.2 KB |
|
PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏 |
The observations-list agent note told every caller to read citation timestamps from `reasoning_segments`. Summarizer output has no such field; its citation segments are `summary_segments`. The note now names the field for each scanner type. It also claimed the list carries no citation timestamps. That is false for summarizer rows, whose `summary_segments` the new exclude path does not match, so the claim is dropped. Regenerated `services/mcp/src/tools/generated/replay_vision.ts` from `tools.yaml`. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47
…sion The observations-list description said the snapshot prompt and query are the same on every row. They are frozen per observation, and editing either bumps `scanner_version`, so a page written across an edit holds both values. An agent reading that copy attributes older findings to the current prompt. The description now scopes the repetition to one scanner version, says an edit bumps `scanner_version`, and tells a caller spanning versions to read one observation per version before comparing findings. Regenerated the MCP tool schema JSON from `tools.yaml`. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47
Citation segments are written as `{citation_field}_segments`, and a
summarizer cites `summary`, so its duplicate lives in
`summary_segments`. The exclude path matched only `reasoning_segments`,
so a summarizer page still shipped each body twice. That is the scanner
type with the largest free text, so the page-size reduction this change
targets missed it.
Adds the `summary_segments` exclude path, and a summarizer fixture that
proves the row keeps `summary` and loses `summary_segments`. The
existing fixtures are all monitor rows, so nothing covered the second
field name. The test fails without the exclude path.
The description now names the dropped segment field per scanner type.
Regenerated the MCP handler and tool schema JSON from `tools.yaml`.
Generated-By: PostHog Desktop
Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47
…ieve The bundled `exploring-replay-vision-observations` skill names `vision-scanners-observations-list` as the workhorse, then reads citation segments straight off those rows. Trimming the segments off that page leaves the skill prescribing a field the tool no longer returns, so an agent following it drops the deep links the section calls the point of the step. The section now says the scanner-scoped list leaves the segments off, and to read the observation with `vision-scanners-observations-get` before citing it. It also names the segment field per scanner type, and records that the session-scoped tools still return segments, so that path needs no extra call. The skill is seeded per team as an external companion skill, so scout runs load this text. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the |
|
This PR was closed due to lack of activity. Feel free to reopen if it's still relevant. |
Problem
vision-scanners-observations-listat its documented default. The page overflows the exec token cap, so every caller has to guess a smalllimitfirst.scanner_snapshot.scanner_config.promptscanner_snapshot.queryscanner_result.model_output.reasoning_segmentsreasoning, interleaved with timestamp chipsChanges
vision-scanners-observations-getfor one observation's full prompt and citation timestamps, so deep-linking a moment still works, at one extra call per finding.CitedTextreasoning and seekbar citation marks), and the "copy all observations" clipboard export. The REST API keeps serving every field.services/mcp/schema/*.json,src/tools/generated/replay_vision.ts, and the tool-schema snapshot fromtools.yaml.How did you test this code?
services/mcp/tests/unit/vision-scanner-observations-list-trimming.test.tsdrives the generated handler over a mocked API response. It catches a regression no existing test covers: the exclude paths silently no-op if a path stops matching the response shape, which is exactly how this defect would return. The schema-snapshot test only sees the input schema, andtests/unit/utils.test.tsonly covers theomitResponseFieldshelper in isolation.tests/unit/vision-scanner-observations-list-trimming.test.ts,tool-schema-snapshots,generate-tools, andutilsinservices/mcp, pluslint-tool-namesandhogli ci:preflight --fix(no failures).pnpm typecheckinservices/mcpcannot run clean in this sandbox because the nested@posthog/quillworkspace is unbuilt, which errors on every product's MCP app. No error referenced the two files changed here.Automatic notifications
Docs update
None. The tool description and agent note are the user-facing surface, and both are updated here.
🤖 Agent context
Autonomy: Fully autonomous
Written by Claude Code from a PostHog inbox report. Skills invoked:
/writing-simplified-technical-english,/writing-pr-descriptions. Repo conventions for MCP tools came fromservices/mcp/CONTRIBUTING.mdanddocs/published/handbook/engineering/ai/implementing-mcp-tools.md.Two approaches were tried before this one. The first reapplied #68360 in the DRF serializer, gated on
self.action == "list". Tracing the frontend consumers showed that trims real features off four app surfaces, so it was reverted. Theresponse.excludemechanism intools.yamlreaches only agent callers and is what the earlier list-overflow fixes used, so the app is untouched by design.Two details in the report needed correcting on the way. It named
vision-scanners-observations-retrieve; the tool isvision-scanners-observations-get. It also did not account for the tool's agent note, added after #68360 was written, which tells agents to readreasoning_segmentsoff list rows to deep-link a finding. Dropping that field without rewriting the note would have left the tool documenting a field it no longer returns, so the note now describes the retrieve hop.No duplicate:
gh pr list --state open --searchover both the tool name and "replay vision observations list" found no open PR fixing this. #89485 is adjacent (frontend reasoning hover) and does not touch the MCP layer.Public artifact: the report and the live reproduction both involved real customer session content. Nothing from either reached this PR. The test fixture was written from a list of the properties it has to exercise, with the real payload closed, using an invented scanner and invented prose.
Created with PostHog Desktop from this inbox report.